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AFFIDAVIT UNDER 37 C.F.R. §1.131 



Applicant provides this Affidavit in response to the Office action dated November 15, 



2005 in the above-identified patent application. By this Affidavit, the assignee of the above- 
identified patent application (hereinafter the, "Assignee") provides evidence to establish invention of 
the subject matter of the rejected claims prior to the effective date of the reference on which the 
rejection is based. Specifically, the assignee provides evidence of conception of the invention prior to 
the effective date of one of the cited references (Zhou et al.) coupled with evidence of due diligence 
from prior to that effective date to a subsequent reduction to practice or filing date of the above- 
identified patent application. The Assignee submits the following evidence: 



1. Claims 1, 5-21, 25-48, 51-58 and 61-72 currently stand rejected under 35 
U.S.C. § 103 as unpatentable over Krug in view of Zhou et al. Further, claims 3, 4, 
23, 24, 49, 50, 59 and 60 currently stand rejected under 35 U.S.C. § 103 as 
unpatentable over Krug in view of Zhou et al. and Annis. Claims 1, 21, 41, 48, 58 
and 68 are independent claims. 

2. U.S. Patent Application Publication No. US 2004/0213378 to Zhou et al. has 
an effective date as a § 102(e) reference no earlier than April 24, 2003, its effective 
filing date. 
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3. Claim 1 recites: 

A system for detecting an explosive within an article, comprising: 

an acquisition subsystem including an x-ray computed tomography scanner 
having a stationary radiation source and a stationary detector, said acquisition 
subsystem is adapted to acquire intensity measurements pertaining to the explosive; 
and 

a reconstruction subsystem, in communication with the acquisition subsystem, 
for generating view data from the intensity measurements and for reconstructing the 
view data into image data representative of the explosive, wherein said reconstruction 
subsystem utilizes three-dimensional reconstruction techniques. 

4. Appendix 1 includes a power point presentation, dated March 31, 2003, in 
which the claimed invention, including each of its elements, is discussed. Specifically, 
with reference to at least slide 3 of the power point presentation, there is mention of 
"Low-dose sCT", which is nomenclature for a stationary CT system that includes a 
stationary radiation source as well as a stationary detector. On that same slide is the 
mention of several "critical to quality" characteristics of such a CT system, including 
the ability to scan luggage, which is a multi-generational product development 
(MGPD) of Assignee. Further, on at least slide 7, there is discussion of an explosive 
detection system for possible development by Assignee for the Transportation Security 
Administration. 

5. The power point presentation in Appendix 1 also includes a discussion 
beginning at slide 30 of reconstruction techniques under development. Specifically, in 
slide 33, there is mention of adaptive filtering reconstruction, which is done in three- 
dimensions. 

6. Appendix 1 further includes several slides indicating the work to be done to 
reduce to practice the invention recited in claim 1. Specifically, at least slides 4, 8, 9, 
13, 29 and 40 provide an overview of tasks to be performed to reduce to practice the 
invention recited in claim 1. 



2 



Appl. No. 10/743,195 

Reply to Office action of November 15, 2005 



139681-2 



7. Assignee possesses additional evidence to support its conception of the 
invention recited in claim 1 prior to April 24, 2003 and due diligence until at least the 
filing of the above-identified patent application upon request of the Examiner. 

8. Claim 21 recites: 

A system for detecting an explosive within an article, comprising: 
a transportation means for transporting the article; 

an acquisition subsystem comprising an x-ray computed tomography scanning 
device having a stationary radiation source and a stationary detector and being adapted 
to acquire intensity measurements that can be generated into view data pertaining to 
the explosive; 

a reconstruction subsystem, comprising a plurality of reconstruction stages, for 
reconstructing the view data into image data representative of the explosive, wherein 
said reconstruction subsystem utilizes three-dimensional reconstruction techniques; 
and 

a computer-aided detection subsystem, comprising a plurality of computer- 
aided detection stages, for analyzing the image data. 

9. At least slide 2 in Appendix 1 illustrates a transportation means for 
transporting an article. Further, the evidence on conception and reduction to practice 
posed above regarding claim 1 is equally applicable to claim 21 as it applies to the 
acquisition subsystem and the reconstruction subsystem. 

10. Appendix 2 includes an email sent March 26, 2002 from Ricardo Avila, one of 
the inventors of the above-identified patent application. The email message indicates 
that the attachment to the email (also in Appendix 2) is a long term explosive detection 
system proposal including the computer-aided detection subsystem element. Please 
note that Figure 1 of the proposal is very similar to Figure 1 of the above-identified 
patent application. Further, Figure 7 of the attachment shows a stack of CT slices that 
have been processed with volume rendering techniques to create a three-dimensional 
image of the interior of a piece of luggage. 

11. Appendix 3 includes a screen shot of the source code repository at Assignee's 
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Global Research Center in Niskayuna, New York. The screen shot indicates that the 
source code "vtklmageHessianResponse.cxx", which implements a Hessian filter, was 
downloaded into the system at least as early as December 27, 2002. This evidence 
indicates that the inventors of the above-identified patent application had in their 
possession a computer-aided detection subsystem for analyzing image data, as recited 
in claim 21, prior to April 24, 2003. 

12. Appendix 4 includes an e-mail, dated January 23, 2003, to Ricardo Avila, one 
of the inventors of the above-identified patent application, and referring to a Final 
Report sent to L-3, then a customer of Assignee. The e-mail states "The Vector 
Hessian (VHess) algorithm has detection rates that are comparable to but slightly 
lower than the EDS version 104 software". The significance of this e-mail and this 
statement is that they evidence that the inventors had incorporated the computer-aided 
detection subsystem with an acquisition subsystem and a reconstruction subsystem 
prior to April 24, 2003. 

13. Assignee possesses additional evidence to support its conception of the 
invention recited in claim 21 prior to April 24, 2003 and due diligence until at least the 
filing of the above-identified patent application upon request of the Examiner. 

14. Claim 41 recites: 

A system for detecting an explosive within an article, comprising: 

an acquisition subsystem including an x-ray computed tomography scanner for 
acquiring intensity measurements pertaining to the explosive; 

a reconstruction subsystem, in communication with the acquisition subsystem, 
for generating view data from the intensity measurements and for reconstructing the 
view data into image data, wherein said reconstruction subsystem utilizes three- 
dimensional reconstruction techniques; 

a computer-aided detection subsystem for analyzing the image data; and 

at least one additional source of information pertaining to the explosive, 
wherein the image data and the at least one additional source of information assist in 
identifying the explosive. 
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15. The evidence on conception and reduction to practice posed above regarding 
claim 1 is equally applicable to claim 41 as it applies to the acquisition subsystem and 
the reconstruction subsystem. Further, the evidence on conception and reduction to 
practice posed above regarding claim 21 is equally applicable to claim 41 as it applies 
to the computer-aided detection subsystem. 

16. Further, Appendix 1 includes, in at least slide 3, a discussion of energy- 
discrimination computed tomography (EDCT), which is included as an additional 
source of information as recited in claim 41. Furthermore, in at least slide 29, the task 
of EDCT benefiting sCT is mentioned as a deliverable in the third quarter of 2003, 
thus evidencing due diligence to reducing EDCT in sCT to practice. 

17. Assignee possesses additional evidence to support its conception of the 
invention recited in claim 41 prior to April 24, 2003 and due diligence until at least the 
filing of the above-identified patent application upon request of the Examiner. 

15. Claim 48 recites: 

A method for detecting an explosive within an article, comprising: 

acquiring information pertaining to the explosive with an acquisition apparatus 
having an x-ray computed tomography scanner with a stationary radiation source and a 
stationary detector; and 

reconstructing an image representative of the explosive based upon the 
acquired information, wherein said reconstructing includes reconstructing the acquired 
information into a three-dimensional image. 

18. The evidence on conception and reduction to practice posed above regarding 
claim 1 is equally applicable to claim 48 as it applies to the acquiring information with 
an x-ray computed tomography scanner having a stationary radiation source and 
stationary detector and reconstructing a three-dimensional image. 

19. Assignee possesses additional evidence to support its conception of the 
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invention recited in claim 48 prior to April 24, 2003 and due diligence until at least the 
filing of the above-identified patent application upon request of the Examiner. 

20. Claim 58 recites: 

A method for detecting an explosive within an article, comprising: 

acquiring information pertaining to an object located within the article with an 
x-ray computed tomography machine having a stationary radiation source and a 
stationary detector; 

communicating the acquired information to a plurality of reconstruction 
modules; 

reconstructing the acquired information into image data with the plurality of 
reconstruction modules, wherein said reconstructing includes reconstructing the 
acquired information into a three-dimensional image; and 

analyzing the image data to identify whether the object is an explosive device. 

21. The evidence on conception and reduction to practice posed above regarding 
claim 1 is equally applicable to claim 58 as it applies to the acquiring information with 
an x-ray computed tomography scanner having a stationary radiation source and 
stationary detector and reconstructing a three-dimensional image. 

22. Further, the information provided in Appendix 4 is significant in that it 
evidences that the inventors had incorporated the computer-aided detection subsystem 
with an acquisition subsystem and a reconstruction subsystem prior to April 24, 2003, 
thereby communicating the acquired information and analyzing the image data as 
recited in claim 58. 

23. Assignee possesses additional evidence to support its conception of the 
invention recited in claim 58 prior to April 24, 2003 and due diligence until at least the 
filing of the above-identified patent application upon request of the Examiner. 

24. Claim 68 recites: 

A method for detecting an object, comprising: 
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scanning an article with an x-ray computed tomography machine to acquire 
information pertaining to the object, wherein the computed tomography machine 
includes a stationary radiation source and a stationary detector; 

discriminating between high-energy and low-energy signatures; 

reconstructing image data representative of the object based upon the high- 
energy and low-energy signatures, wherein said reconstructing includes reconstructing 
the information derived from the high-energy and low-energy signatures into a three- 
dimensional image; and 

analyzing the reconstructed image to identify the object. 

25. The evidence on conception and reduction to practice posed above regarding 
claim 1 is equally applicable to claim 68 as it applies to scanning an article with an x- 
ray computed tomography scanner having a stationary radiation source and stationary 
detector and reconstructing a three-dimensional image. Further, the evidence on 
conception and reduction to practice posed above regarding claim 41 is equally 
applicable to claim 68 as it applies to discriminating between high and low energy 
signatures and as it applies to reconstructing information derived from high and low 
energy signatures. 

26. Assignee possesses additional evidence to support its conception of the 
invention recited in claim 68 prior to April 24, 2003 and due diligence until at least the 
filing of the above-identified patent application upon request of the Examiner. 

27. Assignee submits that, in accordance with the above remarks and the 
accompanying Appendices, it has established conception of the invention of the 
subject matter of the rejected claims prior to April 24, 2003 and, at the very least, due 
diligence until constructive reduction to practice, namely the filing of the above- 
identified patent application. 

28. All statements made by the declarant herein that are based on personal 
knowledge are true and all statements based on information and belief are believed to 
be true. These statements are made with the knowledge that willful false statements 
and the like so made are punishable by fine or imprisonment or both, under section 
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1001 of Title 18 of the United States Code, and that such willful false statements may 
jeopardize the validity and/or enforceability of the application or any patent issuing 




Patent No. 39,803 
Supervisory Patent Attorney 
General Electric Company 
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Appendix 2 



f 

Powell, William (GE, Research) 



From: Leue, William M (GE, Research) 

Sent: Friday, January 27, 2006 2:24 PM 

To: Powell, William (GE, Research) 

Subject: CAD integrated with recon for luggage screening 

Importance: High 



Bill, 



The forwarded email is the earliest reference I can find: 3/28/2002. It encloses a document that Rick Avila wrote that 
- includes words like '...tight integration between recon and CAD...'. 



-Bill 



— Original Message — 
From: Avila, Ricardo S (CRD) 

Sent: Thursday, March 28, 2002 8:49 AM 

To: Leue, William M (CRD) 

Subject: FW: CAD/Vis Section 



Take a look at the CAD/Vis luggage screening document for some background. After yesterday's discussion with 
L3 we now have to revise the CAD section to not assume we will be doing full blown modeling based on shape (shoe, hard 
drive, etc.) I will explain later today. 

- Rick 

— Original Message — 



From: Avila, Ricardo S (CRD) 

Sent: Tuesday, March 26, 2002 5:08 PM 

To: Mundy, Joseph (CRD); McCulloch, Colin (CRD); Kaucic, Robert A (CRD); Lorensen, William E (CRD); Kelliher, Timothy P (CRD) 

Cc: Grabb, Mark L (CRD) 

Subject: FW: CAD/Vis Section 



Here is the CAD/Vis part of the EDS long term proposal (5 years). Please give it a read and mark up changes in 

red. 

We will also need to write up a short term CAD/Vis proposal that I am now told will be for 3 years (not one year as 
we originally thought). 

- Rick 



— Original Message — 



From: Avila, Ricardo S (CRD) 

Sent: Friday, March 22, 2002 3:07 PM 

To: 'jerrybrooks@pagesplusinc.cofTV; Hopkins, Forrest (CRD) 

Cc: Walter, Deborah (CRD) 

Subject: CAD/Vis Section 



Here is the final CAD/Vis input for our 3pm deadline. 

I will also put this into the quickplace under EDS - Long Term. 

Let me know if you see an issue with it. 




5-CADVisProposal 
3_22 L 02.doc 



1 



Ricardo S. Avila 

Clinical Applications and CAD Project Manager 
Imaging Technologies 

GE Global Research Center Email: avila@crd.ge.com 

1 Research Circle, KW C220A Phone: (518) 387-6632, 8*833-6632 

Niskayuna, NY 1 2309 Fax: (51 8) 387-6981 



CAD/Visualization Technical Discussion and Approach 

1.0 CAD/Visualization System Architecture 

The next generation of Explosive Detection Systems must leverage and globally optimize 
across a variety of information, threat sensing, and baggage handling subsystems. The 
information obtained from these subsystems will be processed by state-of-the-art 
Computer Aided Detection (CAD) and Visualization subsystems. The main goal of the 
CAD and Visualization Subsystems is to achieve a high baggage throughput, a high 
detection rate (sensitivity), and a low false alarm rate (1 -specificity) with as little human 
assistance as possible. In addition, we seek an EDS design that can easily and 
appropriately scale to the capacity, and level of threat measurement and processing 
required at an individual airport given rapidly changing security concerns. 

Achieving these goals for an EDS with a 1500 bag/hour throughput translates into an 
average CAD processing time of approximately 2 seconds per scanned bag. A high 
throughput and automated EDS that achieves high detection accuracy will need to: 

(1) Consider as much salient information as is available on the passenger and 
baggage in question. Significant gains are possible leveraging a dual energy 
CT acquisition system and an ETD monitor. 

(2) Tightly couple and pipeline the CT reconstruction and CAD subsystems in 
order to streamline data processing and automatically obtain the highest 
possible image quality when the level of threat is found to be high. 

(3) Analyze the contents of each bag with state-of-the-art Computer Aided 
Detection algorithms that comprehensively model False Alarm objects as well 
as explosive devices and optimally reason with all of the information 
available. 

(4) Provide all relevant information from all subsystems to a highly automated 
visualization application console for a final digital review of suspicious 
baggage by a human operator. 

(5) Utilize a powerful, cost effective, and scalable computing engine that 
adaptively allocates processing power to the level of threat. 

Figure 1 illustrates the flow of baggage and information across such a system. Those 
elements shown in green can be considered optional modules which when added to the 
EDS would significantly improve detection rate and reduce the false positive rate. 



CT 
Acquisition 
Subsystem 
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Figure 1: A highly integrated Reconstruction, CAD, and Visualization Architecture 

The system starts with a high throughput stream of baggage flowing into the CT 
acquisition subsystem. The acquired projection data is incrementally transferred to the 
reconstruction subsystem for processing into a 3D CT representation. The 3D 
representation will ideally consist of two scalar CT values representing two independent 
energy levels. Each reconstructed slice is incrementally then sent to the CAD subsystem 
for automatic threat detection. If passenger and flight information is available for each 
bag then this will also be provided to the CAD subsystem. An optional ETD subsystem 
may also scan the bag for trace evidence of explosives. The ETD information including 
the level of threat and the chemical composition of the sampled threat will also be 
provided to the CAD subsystem. The CAD subsystem performs advanced image analysis 
on the 3D CT scans of the luggage taking into account all of the information provided on 
the passenger, the flight, and the sensors applied to the baggage. This would allow the 
EDS to automatically customize the level and type of scrutiny placed on an individual 
bag based on all of the information available. 

The Reconstruction and CAD subsystems are tightly coupled allowing for additional 
targeted reconstructions to be automatically performed on baggage if the CAD subsystem 
requires higher quality slice data to make a critical decision. Both subsystems will utilize 
a scalable cluster of standard PC processing units. This novel architecture allows the 
Reconstruction and CAD subsystems to dynamically adjust computational loading for 
each piece of luggage. The reconstructed images will then be sent to a hierarchical CAD 
subsystem consisting of three levels of CAD processing. Should the CAD subsystem find 
that the bag is a significant threat to the safe operation of the aircraft it must send a signal 
to have the bag automatically pulled from the baggage stream and held for additional 
scrutiny. The low percentage of bag scans that are considered suspicious after CAD 



processing will be analyzed by human operators utilizing advanced imaging tools in the 
Visualization Subsystem. This last level of digital inspection is intended to minimize the 
number of bags that must be manually opened and inspected. All of the information 
acquired and computed for the bag will then be sent to a Visualization subsystem where 
operators will have advanced 2D and 3D tools for performing a virtual inspection of the 
suspicious bag. If after operator analysis the bag is no longer considered a threat it will be 
placed back in the normal baggage stream. Otherwise it will be sent to a holding area for 
manual inspection. The performance of the entire EDS will be optimized in order to 
achieve the best combination of operations cost and accuracy of the system. 

1.1 General CAD Approach 

GE Global Research has developed a general approach to the Computer Aided Detection 
(CAD) of objects based on the use of mathematical models. These models represent 
various stages of representation for CT images of a subject. At present, the approach is 
based on three stages of representation and is being applied toward the early detection of 
lung cancer in CT images. The models are currently directed at the representation of the 
lung in support of lung cancer detection in CT images, however the approach can apply 
to any detection application. 

Signal models - The first stage of representation is the signal formation process. In this 
example, the images are formed by X-ray computer tomography (X-ray CT). The signal 
models represent the blurring and noise process inherent in the CT image formation 
process. These models also support the detection of primitive events in the image such as 
linear features or discontinuities in density. These primitive features form the basis for 
later stages of representation. These signal models utilize detailed knowledge of the 
physics of image formation and the resulting signal properties. 

Shape models - The next stage of representation captures the appearance of features in 
the image in terms of their intensity and shape. This representation provides an 
elementary vocabulary for describing three dimensional structures and to group the 
detections made at the signal level. The role of these representations is to explain the 
intensity or density of groups of pixels in terms of perceptual units. 

Object models - The last stage of representation is in terms of physical object concepts. 
The earlier representations are further grouped and parameterized in terms of higher level 
object units. For example, a sheet object along with a thin cylindrical wire can be 
interpreted as an explosive threat with appropriate constraints on the density and mass 
that would be consistent with explosive manufacturing technology. 

Model Selection - At any stage of representation there are a number of models that 
might provide an interpretation of the underlying image or group of features. It is 
necessary to have a unified framework for model selection. In our current approach we 
use Bayesian statistical methods to select the model that provides the most probable 
interpretation of the data, given prior constraints on the model. The models are viewed as 
competing for the best interpretation of the data. In this approach, there is no need for 



pattern recognition or neural net classifiers since the models carry their own classification 
mechanisms that are particularly tailored to the data representation. 



Figure 1 demonstrates our early work in automated detection of lung cancer in low dose 
CT scans. The left image (a) demonstrates the ability to automatically detect the outer 
lung boundary in CT scans. The middle image (b) shows the ability to track vascular 
structure (blue) as well as candidate regions in the image that may indicate the early 
presentation of lung cancer (green). The right image (c) is a magnification of detection 
results centered on a suspicious lung lesion. 




Figure 1 Examples from our current work on lung cancer. 



GE's Complete knowledge of medical scanner acquisition and reconstruction techniques 
will allow us implement carefully engineered signal models that avoid the need for 
learning or training. The models can be adapted by known acquisition parameters or 
from data from a single phantom. At the same time, our sound mathematical modeling 
effort will enable quantitative analysis of the effect of acquisition protocols on detection 
performance. For example, in acquisition based on X-rays, it is necessary to trade off 
dose against detection sensitivity. 



1.2 CAD Subsystem 



To achieve our three goals of rapid luggage throughput, extremely high sensitivity for 
explosive detection, and a low false positive rate we propose a hierarchical architecture 
for Computer Aided Detection (CAD). Our system will consist of several levels of 
processing. We rule out clearly nonthreatening luggage early in the hierarchy with a 
minimum amount of processing. In subsequent levels, we employ an amount of 
processing which is commensurate with the perceived level of threat. In this way, for a 
given luggage throughput specification, our ability to rule out luggage through coarser 
image processing methods allows us more processing time for finer detection techniques. 
Each processing level will maintain a high specified detection sensitivity level. 

We propose a three level CAD system. Each level of processing will be based on the 
same overall Model-Based Detection framework, where the sophistication of the models 
and data increases as we descend the hierarchy. We define Model-Based Detection as the 
identification and classification of objects through the use of detailed models fusing CT 



density, morphology, and domain knowledge quantified in the form of prior probability 
distributions. These models can be further supplemented with other non-image 
information to increase detection performance. 

The aim of the first level of detection is to rapidly screen baggage that can be ruled out 
with computationally inexpensive image processing methods. The result of this first 
processing level is high sensitivity and relatively low specificity. For instance, large 
masses can be formed by connecting regions of similar CT density to form geometric 
structures. Baggage scans containing masses exhibiting properties consistent with 
explosive materials are passed on to the second level of processing, while those bags not 
exhibiting such properties require no further analysis. Sensitivity will be significantly 
increased at this and later detection processing levels when dual X-ray energy 
discrimination is achievable. We plan to assign a significant amount of effort to this task. 

In the second level of detection processing, more computational resources are focused on 
the potentially suspicious luggage scans to achieve higher specificity without sacrificing, 
sensitivity. Here, more detailed differential geometry-based segmentation can be used to 
permit finer identification of objects. In our many years of experience developing 
automated detection systems for intelligence, military, and medical applications, we have 
found that differential geometric image analysis methods provide a robust and highly 
sensitive framework for image quantification. For example, the three-dimensional 
Hessian matrix (the 3x3 matrix of spatial second order partial derivatives) gives several 
measures for quantifying object shape from intensity profiles. Figure 3 gives a volume 
rendering of the vessel network in a CT lung scan. In this application, we segmented the 
lung's vessel network to improve cancer detection. Here we plot the square root of the 
product of the first two Hessian eigenvectors in shades of red. This measure is designed 
to highlight regions of cylindrical (vessel-like) intensity profiles, and it performs 
extremely well in this application. This has direct application to explosives detection 
because a variety of different canonical object shapes (for instance planar or sheet-like 
explosives) can be identified in the CT imagery through properly chosen functions of 
differential geometric operators. 



Figure 3: Three-dimensional volume rendering of the vessel network in a CT lung scan. The square 
root of the product of the first two eigenvalues of the 3d Hessian matrix (H) is given in red. Each 
column of the matrix V contains an eigenvector, vO, vl, and v2 which are oriented relative to a 
cylinder as shown in the right panel. 




A key advantage we have at GE when designing segmentation and detection algorithms 
is our ability to use comprehensive knowledge of the point-spread function characteristics 
of the CT scanner to aid precise identification of objects. For example, further processing 
of the lung scan given above was performed to accurately measure the fine structure 
around air bronchioles. These tiny airways have wall thicknesses on the order of the 
resolution of the CT scanner. We must therefore carefully use the scanner specifications 
when making these subpixel measurements. The top panel of Figure 4 gives a graphical 
representation of our bronchiole wall CT profile model. This model involves a 
convolution of an anatomical bronchiole model with the known scanner point-spread 
function to generate a predicted CT intensity profile. The bottom panel shows our 
detections of bronchioles in a sample image. The wall thickness is measured by 
deconvolving the scanner point spread function from the anatomical model and 
estimating the model's thickness parameter via statistical maximum likelihood 
estimation. 



Figure 5: Bronchiole wall detection and measurement. The left panel gives the convolution of 
anatomical model with the scanner point spread function. The right panel gives an example lung 
scan with two automatic detections highlighted. 

Detailed probabilistic models will be developed of both threat and non-threat objects. 
These models will be constructed from a collection (database) of example imagery of 
explosive materials and confounding objects. We will classify segmented objects (called 
"model matching" in our task list) by incorporating features obtained from the imagery, 
chemical trace analysis, and passenger and flight information using pattern recognition 
methods. In our experience, we have found optimal Bayesian classification methods to 
be most powerful in discerning targets of interest (e.g. explosives) from confounding 
objects (those articles expected in normal luggage). This type of classification proceeds 
by first assigning prior probability distributions on object models using expert knowledge 
on target characteristics like CT signature, morphology, and any available non-image 
data. Next, we combine these models with the incoming data on a particular bag through 
strict Bayesian probability calculations to report an overall probability of explosives 
threat. Bags exhibiting a specified minimum threat probability are deemed suitable for 
further scrutiny. 

Baggage not ruled out as nonthreatening continues to level three detection. Objects 
deemed suspicious by level two processing are reconstructed at a higher resolution using 
the reconstruction hardware. In addition, reconstruction artifacts due to baggage 
hardware identified in level two are eliminated. Classification is then performed on this 
higher resolution, artifact-free imagery resulting in higher classification rates of 
suspicious objects. This image reconstruction feedback loop can be implemented 
iteratively, by continuously identifying suspicious objects through Bayesian classification 
and increasing CT resolution at those object locations. In this way we constantly 
optimize our certainty (probability) of candidate object model representations. 

Finally, bags that exhibit a high threat probability by the hierarchical CAD system and 
therefore are deemed suspicious are sent to a visualization workstation for interactive 
human review. A key strength of our model-based processing pipeline is that the salient 



detection probabilities and other segmentation measures are sent to the visualization 
workstation to automate and guide interactive inspection. 

For development of sensitive and robust CAD systems, a large database of imagery and 
metadata will be required. Ultimately this database will contain labeled imagery of 
threats and non-threats from the final production CT hardware with chemical trace and 
other non-image data. Initially, however, labeled imagery from currently available CT 
systems will be acquired for CAD development. This task will require detailed work by 
CT technicians and screening experts to formulate a representative database of the full 
range of luggage scenarios. 

1.3 Detection System Validation 

GE Global Research has developed a fully automated infrastructure for detection 
algorithm testing and validation consisting of several components. Software quality is 
monitored through nightly builds and automatic algorithm tests. A test-bed cluster of 92 
Pentium class CPU's is managed by a web-enabled interface. This facilitates algorithm 
developers to design experiments which measure performance of proposed algorithms on 
a large database of images with minimal interaction. We quantify algorithm performance 
statistically using general Receiver Operating Characteristic Curves (ROC) and other 
performance summary measures. ROC curves are effective at showing graphically the 
trade-off between CAD system sensitivity and specificity. Also, ROC analysis allows us 
to statistically compare the performance of two candidate detection systems to determine 
if perceived performance differences are real or merely due to noise in the test image 
database. 

Receiver Operating Characteristic Curve 




0 0.2 0.4 0.6 0.8 1 
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Figure 6: An example receiver operating characteristic curve comparing two CAD algorithms. The 
algorithm plotted in pink gives consistently higher detection sensitivity for a range of false positive 
rates (specificity). 

The significant time GE has devoted to this validation infrastructure gives us a large 
head-start to begin testing detection methods at the very beginning of an awarded 
contract. 



1.4 Visualization Subsystem 

Visualization is the process of transforming and combining data into forms that are 
suitable for presentation to a human. In the baggage inspection system, the visualization 
station is the vehicle for displaying the fused information from various sensor and 
analysis subsystems. The system serves as the interface between the human operator, the 
imaging equipment and the segmentation and detection algorithms. The major challenges 
of the visualization system revolve around the baggage system's high throughput 
requirement. To achieve the desired throughput the visualization must be able to give the 
operator an initial pass/fail indication at a glance. For those that are indicated as failing, 
the visualization must be tuned to display the reasons for failure and provide a means for 
further interrogation. This interrogation must be intuitive in its application and self- 
guiding such that the system operator does not spend time interpreting the unprocessed 
CT imagery. Thus speed, rapid navigation of the 3D volume, fusion of acquired and 
derived information, limited user input and interaction become the driving requirements 
for the system operational concept. To meet these requirements, we propose to apply 
existing and develop new visualization algorithms augmented by automated parameter 
selection and adaptive visualization protocols. These techniques will be provided in a 
baggage visualization toolkit that is designed to achieve these requirements - and be 
flexible enough to continuously improve as new techniques and customer-specified 
display platforms evolve. 




The acquired stack of CT slices can be visualized all at once using volume rendering 
techniques. Volume rendering displays "projections" of the slices. These projections are 
synthesized by passing rays through the volume elements (voxels) of the data. The 
projection can simulate physical properties such as x-ray attenuation. However, many 
other projections are possible by remapping the sampled values through "transfer 
functions." The transfer functions convert the samples and other local information into 
intensities, colors and opacities. Through appropriate transfer function design, CT data 
can be transformed into displays that highlight various materials as shown in Figure 7. 



However, as with two-dimensional displays, transfer function design is too difficult and 
time-consuming to be used in a high throughput application. The degree of difficulty is 
potentially higher in baggage systems than in medical applications since the range of 
materials in a suitcase is so vast. 

The main focus of the volume display is performance and automation. Performance of the 
volume rendering has a significant impact on system throughput. We will develop 
techniques that produce the volume rendering as soon as become available from the 
acquisition and detection subsystems. This incremental volume rendering approach 
permits overlapping the volume presentation with reconstruction and analysis. Once the 
final slices are available the rendering is complete. We have successfully deployed real- 
time incremental volume rendering and analysis on our Lightspeed helical CT system. 
We will also investigate other volume acceleration techniques including hardware 
acceleration using commodity graphics boards and distributed rendering across multiple 
commodity workstation systems. 

Automation impacts operator performance and effectiveness. We propose to investigate, 
invent and implement automated transfer function technology that will highlight all 
important structures and materials found in the baggage. These novel techniques will 
produce transfer functions that map single or dual energy intensities and image gradient 
information into effective presentations of significant objects in the baggage. The results 
of the detection and segmentation algorithms will be used to guide the selection and 
parameterization of the transfer functions. For example, we have already demonstrated 
techniques that use volume segmentation masks to apply multiple transfer functions 
within a volume. This mechanism provides visual emphasis to the results created by the 
detection subsystem. 

Detection and segmentation results identify regions of the CT data that warrant further 
processing. Typically, segmentation algorithms produce labels that identify a voxel as 
being a member of a set of possible objects. Detection algorithms produce a wealth of 
quantitative information about regions of the volume. The detection results may be voxel 
labels but are more often defined by line segments or surfaces. 

We propose to integrate (or fuse) the region and surface detection results with the 
intensities in the CT volume. The detection results will be used to emphasize or de- 
emphasize regions of the volume displays. For example, the results of segmentation may 
identify regions that are known to be non-threats. The visualization system can use this 
information to remove those structures from the presentation. We have demonstrated this 
capability in a medical application that removes the bone from CT abdominal studies so 
that the vascular structure is easy to visualize. 

Simulation results include geometric models of baggage and baggage contents as well as 
volumetric models of these items and explosive materials. These models can be 
registered with the scanner results to help operators perform visual comparisons with 
known threats and non-threats. For example, a volume model of a particular brand of 
suitcase can be juxtaposed with the scans of the passengers luggage. Suspicious 



intensities in or around the plastic wheels can be visually compared with the known scans 
of the reference luggage. We propose to provide registration techniques that will 
automatically align simulation models with the acquired data. 

The visualization algorithms will be encapsulated in a toolkit that can operate on both 
Unix/Linux and Windows platforms. This approach permits the integration of the 
techniques into a various display systems. The architecture of the toolkit will permit the 
addition of new visualization techniques as they are discovered. 

1,5 Development Plan 

Task 1.1 - Establish Database 

We will develop a development database of CT scans for CAD and Visualization 
algorithm development and validation. A large number of scans must be acquired 
to obtain statistically significant validation results. The scans must contain objects 
that represent the full population of true explosive devices as well the many 
objects that may cause a false alarm. In addition the development of phantoms 
will be required. This task will be frontloaded to provide the algorithm developers 
with sufficient data to begin designing algorithms as early as possible. 

Task 1.2 - Scanner Modeling 

This task will focus un obtaining a thorough understanding of the CT scanner 
point spread function and all factors that will influence it. In particular the point 
spread function of the scanner must be well understood in order to achieve 
optimal detection accuracy. 

Task 1.3 - Object Segmentation 

This work will center on the analysis on the generation of a fully automatic and 
three-dimensional segmentation of the 3D CT scan using 3D edge detection 
methods. These regions will be topologically consistent and created incrementally 
as data is provided to the CAD subsystem. 

Task 1.4 - Geometric Representations 

The result of object segmentation will be analyzed to understand the 3D geometry 
of the objects in the CT scan. These techniques must be resilient to significant 
metal artifact and low contrast scanning issues. 

Task 1.5 - Threat/Non-Threat Model Development 

Geometric models are developed into precise mathematical models capturing the 
variation within model populations. Significant interaction time will take place 
between algorithm developers and luggage screening and explosives technology 



personnel to arrive at the best possible models. A large number of threat and non- 
threat models will need to be developed to arrive at high detection sensitivity and 
specificity. Methods must be developed to determine when it is appropriate to 
request a higher quality reconstruction. 

Task 1.6 - Model Matching 

The decision making framework for selecting the most probable model will be 
developed. This framework must support the hierarchical nature of the decision 
making process. The framework will be flexible in that it will allow for updates to 
the number and types of models as new threats and false alarms are discovered. 

Task 1.7 - Hierarchical Detection Architecture 

The computing platform, high level application and data transfer required to 
support a hierarchical CAD architecture will be developed. This system must be 
able to dynamically adjust resources as the level of threat changes. 

Task 1.8 - Optimization of Performance 

Achieving a CAD processing rate of approximately 2 seconds per bag will require 
extensive performance optimization on algorithms and data transport through the 
system. 

Task 1.9 - Energy Discrimination Modeling 

The entire CAD processing architecture will be extended to support dual energy 
discrimination. Since some EDS systems will not have this capability the CAD 
subsystem must be able to leverage the extra energy data when it is available. 

Task 1.10 - 3D Presentation 

2D and 3D rendering techniques will be developed in the context of operator 
decision making ability. Volume rendering algorithms will be developed that are 
capable of displaying multiple scalars fields (dual energy) for maximum operator 
understanding. Algorithms will be developed that automate the generation of 
visualization settings. A luggage visualization toolkit will be developed that 
provides these techniques in a form that is easy to incorporate into applications. 

Task 1.11 - Visualization Graphics Hardware 

Advances in graphics hardware will be leveraged to allow the Visualization 
console operator to manipulate and visualize CT scans in real-time. 

1.6 Milestones and Gaant Chart 
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Task 1 CAD and Image Quality. 

1 .1 Establish database. 

1 .2 Scanner modeling. 

1 .3 Segmentation. 

1 .4 Geometric representations. 

1 .5 Threat/non-threat model development. 

1.6 Model matching. 

1 .7 Terraced detection architecture. 

1 .8 Optimization of performance. 

1 .g Energy discrimination modeling. 

1.10 3D presentation. 

1.11 Visualization graphics hardware. 

Milestone: 35% false positive rate 
(coinciding with first image) achieved. 

Milestone: Less than 20% false 
positive rate achieved. 


















































































































































































































































































































































Task 2 Program Management 

Quarterly reports 
Annua! reports 
Final report. 
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Appendix 4 



Powell, William (GE, Research) 



From: 

Sent: 

To: 



Subject: 



Leue, William M (Research) 
Thursday, January 23, 2003 2:56 PM 

Avila, Ricardo S (Research); Gorman, William (Research); Hammond, Christopher R 
(Research); Kaucic, Robert A (Research); Kelliher, Timothy P (Research) 
Corrected version of Final Report sent to L-3 



Tim and I worked with some data supplied by Eric Cardinal of L-3 to finally resolve the preclassif ier detection rate puzzle. I 
have sent a corrected version of the Final Report to L-3. So far as I am concerned, this action finishes ALL of our 
deliverables for the 2002 project. 

In my email that accompanied the report, I said: 

"In my opinion, the broad conclusions of the report do not change much. The Vector Hessian (VHess) algorithm has 
- detection rates that are comparable to but slightly lower than the EDS version 104 software. The VHess algorithm also 
generates many fewer total sheet objects than does EDS1 04. This fact should make it easier for the classifier to separate 
true detections from false alarms. However, writing a classifier that matches the output of VHess was outside the scope of 
this project, and so realizing the potential performance of the VHess algorithms will require further work." 

I don't want to bomb your mailboxes, but you can get a copy of corrected report from the CVS repository: 
l3det/docs/FinaLreport_v2.doc. 



-Bill 



Program: Visualization Toolkit 

Module: $RCSfile: vtklmageHessianResponse . cxx, v $ 

Language: C++ 

Date: $Date: 2002/12/27 13:54:11 $ 

Version: $Revision: 1.26 $ 



#include "vtklmageData .h" 

#include "vtklmageHessianResponse .h" 

#include "vtkMath.h" 

#include " vtkObj ectFactory . h" 

#include "vtkPointData .h" 



#define IXX VTK_HESSIANRESPONSE__IXX 
#define IXY VTK_HESSIANRESPONSE_IXY 
#define IXZ VTK_HESSIANRESPONSE_IXZ 
#define IYX VTK_HESSIANRESPONSE_IYX 
#define IYY VTK_HESSIANRESPONSE_IYY 
#define IYZ VTK_HESSIANRESPONSE_IYZ 
#define IZX VTK_HESSIANRESPONSE_IZX 
#define IZY VTK_HESSIANRESPONSE_IZY 
#define IZZ VTK_HESSIANRESPONSE_IZZ 
#define I VTK_HESSIANRESPONSE_I 



vtklmageHessianResponse* vtklmageHessianResponse: :New() 
{ 

// First try to create the object from the vtkObj ectFactory 

vtkObject* ret = vtkObj ectFactory : :CreateInstance ( " vtklmageHessianResponse" ) ; 
if (ret) 
{ 

return (vtklmageHessianResponse* ) ret ; 
} 

//If the factory was unable to create the object, then create it here, 
return new vtklmageHessianResponse; 



// 

// Constructor sets default values 
vtklmageHessianResponse: : vtklmageHessianResponse ( ) 
{ 

this->Gammal2 = 1.0; 
this->Gamma23 - 1.0; 
this->Alpha = 0.25; 

this->LowerIntensityThresh = -VTK_LARGE_FLOAT ; 
this->UpperIntensityThresh = VTK_LARGE_FLOAT ; 
this->SetResponseTypeToSheet ( ) ; 
this->VectorData = vtkFloatArray : :New( ) ; 
this->VectorData->SetNumberOf Components (3) ; 
this->ScalarData = vtkFloatArray : :New( ) ; 
this->ScalarData->SetNumberOf Components ( 1) ; 
this->SelectedEigenvector = 0; 

} 



// 

// Destructor cleans up 

vtklmageHessianResponse : : -vtklmageHessianResponse ( ) 
{ 

cout « "vtklmageHessianResponse: : -vtklmageHessianResponse. now entering" << 
endl ; 

this->VectorData->Delete ( ) ; 
this->ScalarData->Delete ( ) ; 

} 

// 

void vtklmageHessianResponse: : Executelnf ormation ( vtklmageData **inData, 

vtklmageData *outData ) 

{ 

outData->SetScalarType (VTK__FLOAT) ; 

} 

// 

typedef float ( *ResponseFunctionPtr) ( vtklmageHessianResponse* self, float 
♦eigenvalue, float **eigenvector ); 

// 

static float vtklmageHessianSheetFunction ( vtklmageHessianResponse* self, float 

♦eigenvalue, float **eigenvector ) 

{ 

float maxL, midL, minL; 
float response; 

// Parameters 

float Gammal2 = self ->GetGammal2 ( ) ; 
float Gamma23 = self ->GetGamma23 ( ) ; 
float Alpha = self->Get Alpha ( ) ; 

// eigenvalues are already sorted by magnitude 
maxL = fabs (eigenvalue [ 0 ]) ; 
midL = fabs (eigenvalue [ 1 ]) ; 
minL = fabs (eigenvalue [2 ]) ; 

//no longer works because signed sorting has been removed 
if (0) { 

//only look at places where the eigenvalues are all negative 
if (maxL >= 0) return 0; 

//float factO = exp(-maxL); 
float factl = fabs ( log ( 1-minL) ) ; 
float fact2 = f abs ( log (midL/minL) ) ; 
//float fact3 = pow( ( 1+midL/minL) , 0 . 5) ; 
//float fact4 = log(-midL); 
//float fact5 = log (midL/maxL) ; 

if (factl*fact2 < 1) return 0; 



response = fabs (log ( factl *fact2 )) ; 
} 



response = maxL - (Gammal2*midL + minL) ; 
if (response < 0.0) { 
response = 0.0; 

} 

return response; 

} 

// 

static float vtklmageHessianLineFunction ( vtklmageHessianResponse* self, float 

♦eigenvalue, float **eigenvector ) 

{ 

float 11, 12, 13; 
// Parameters 

float Gammal2 = self ->GetGammal2 ( ) ; 
float Gamma23 = self ->GetGamma23 ( ) ; 
float Alpha = self ->GetAlpha ( ) ; 

11 = eigenvalue [0] ; 12 = eigenvalue [ 1 ] ; 13 = eigenvalue [2 ] ; 
float 1123 = 0.0; 

if ( 11 <= 0.0 ScSc 12 < 0.0 && 13 < 0.0 ) 
{ 

1123 = fabs ( 13 ) * 

pow ( static_cast<f loat> (12 / 13), Gamma23 ) * 

pow ( static_cast<float>(1.0 + 11 / fabs ( 12 )), Gammal2 ); 

} 

else 
{ 

if ( { fabs ( 12 ) / Alpha ) > 11 && 11 > 0.0 && 12 < 0.0 && 13 < 0.0 ) 
{ 

1123 = fabs ( 13 ) * 

pow ( static_cast<f loat> (12 / 13), Gamma23 ) * 

pow ( static_cast<f loat>(1.0 + 11 / fabs ( 12 ) ) , Gammal2 ); 

} 

else 
{ 

1123 = 0.0; 
} 

} 

return 1123; 



// 

static float vtklmageHessianSphereFunction ( vtklmageHessianResponse* self, 

float ^eigenvalue, float **eigenvector ) 

{ 

// Want to measure how "equal" the eigen values are. They should 
// all be negative and roughly equal. The response should be the 
// smallest in absolute value weighted by a function that decreases 
// with the difference between them, 
float 11, 12, 13; 

11 = eigenvalue [0] ; 12 = eigenvalue [ 1 ] ; 13 = eigenvalue [2 ] ; 

if ( 11 > 0.0 ) 
{ 

return 0.0; 
} 



// Set the weight as a ratio of ( 13 - 11 ) / 12 . 
// will be maximum when 11-12-13 



float w; 

w = ( 13 - 11 ) / 12; 

return -13 * pow (static_cast<f loat> (w) , self ->GetGammal2 ( ) ); 



// 

// Copies eigenvalues from eigenvalue to outvalue, but 
// just changes pointer for outvector 

// avoids expensive copy for each element of eigenvector 
void SortVectorsByMagnitude ( float* eigenvalue, float** eigenvector, 

float* outvalue, float** outvector) 

{ 

float absll = fabs ( eigenvalue [ 0] ) ; 
float absl2 = fabs ( eigenvalue [1] ); 
float absl3 = fabs ( eigenvalue [2 ] ) ; 



if ( absll >= absl2 ) { 
if ( absll >= absl3 ) { 

outvalue [0] = eigenvalue [0] ; 
outvector[0] = eigenvector [ 0] ; 
if ( absl2 >= absl3 ) { 



outvalue [1] 
outvector [1] 
outvalue [2] 
outvector [2] 

} 

else { 

outvalue [1] 
outvector [1] 
outvalue [ 2 ] 
outvector [2] 

} 

} 

else { 

// LI > L2, L3 
outvalue [0] = 
outvector [0] = 
outvalue [1] 
outvector [1] = 
outvalue [2] = 
outvector [2] = 

} 



eigenvalue [1] ; 
eigenvector [1] ; 
eigenvalue [2] ; 
eigenvector [2] ; 



eigenvalue [2] ; 
eigenvector [2] ; 
eigenvalue [1] ; 
eigenvector [1] ; 



> LI 

eigenvalue [2] ; 
eigenvector [2] ; 
eigenvalue [0] ; 
eigenvector [0] ; 
eigenvalue [1] ; 
eigenvector [1] ; 



} 

else 



{ 



// L2 > LI 

if ( absl2 >= absl3 ) { 
// L2 > L3 

outvalue [0] = eigenvalue [ 1 ] ; 
outvectorfO] = eigenvector [1] ; 
if ( absll >= absl3 ) { 
// LI > L3 

outvalue [1] = eigenvalue [ 0] 



outvector [1] 
outvalue [2] 
outvector [2] 

} 

else { 

outvalue [1] 
outvector [1] 
outvalue [2] 
outvector [2] 

} 

} 

else { 

// L3 > L2, L2 
outvalue [0] = 
outvector [0] = 
outvalue [1] = 
outvector [1] = 
outvalue [2] = 
outvector [2] = 

} 

} 

} 



= eigenvector [0] ; 
= eigenvalue [2] ; 
= eigenvector [2] ; 



= eigenvalue [2] ; 
= eigenvector [2] ; 
= eigenvalue [0] ; 
= eigenvector [0] ; 



> LI 

eigenvalue [2] ; 
eigenvector [2] ; 
eigenvalue [1] ; 
eigenvector [1 ] ; 
eigenvalue [0] ; 
eigenvector [0] ; 



// 

// This templated function executes the filter for any type of data, 
template <class T> 

static void vtklmageHessianResponseExecute ( vtklmageHessianResponse *self , 

vtklmageData *outData , 
int outExt[6], int id, 
T* vtkNotUsed ( dummy ) ) 

{ 

int i , j ; 

unsigned long count = 0; 
unsigned long target; 
int myeigenvec; 
int numC; 

int inlncX, inlncY, inlncZ; 
int outlncX, outlncY, outlncZ; 
int maxX, maxY, maxZ; 
int idxC, idxX, idxY, idxZ; 
float 1123; 

ResponseFunctionPtr RFP; 



switch ( self ->GetResponseType ( ) ) 
{ 

case VTK_HESSIAN_RESPONSE_SHEET: 

RFP = vtklmageHessianSheetFunction; 
break; 

case VTK_HESSIAN_RESPONSE_SPHERE: 

RFP = vtklmageHessianSphereFunction; 
break; 

case VTK_HESSIAN_RESPONSE_LINE: 
default : 

RFP = vtklmageHessianLineFunction; 
break; 



T* Hessian[7] ; 

vtkDataObject **InputsDO = self ->GetInputs ( ) ; 
vtklmageData* Inputs[7]; 
float* outPtr; 

myeigenvec = self ->GetSelectedEigenvector ( ) ; 

// Eigen vectors and values 
float *raw_eigenvalue = new float [3] ; 
float **raw_eigenvector = new float* [3]; 
float *eigenvalue = new float [3]; 
float **eigenvector = new float*[3]; 
float **qmatrix = new float* [3]; 
for (i = 0; i < 3; i++) { 

qmatrixfi] = new float [3]; 

raw_eigenvector [i] = new float[3],- 

raw_eigenvalue[i] = 0.0; 

eigenvector [i] = raw_eigenvector [ i ] ; 

eigenvalue [i] = 0.0; 

for (j = 0; j < 3; j++) { 

raw_eigenvector [ i ] [ j ] = 0.0; 

} 

} 

for ( i = 0; i < 7; ++i ) 
{ 

Inputs [ i ] = (vtklmageData* ) InputsDO [ i ] ; 
} 

numC = Inputs [0] ->GetNumberOf ScalarComponents ( ) ; 
// find the region to loop over 
maxX = outExt[l] - outExt[0] 
maxY = outExt[3] - out Ext [2] 
maxZ = outExt[5] - outExt[4] 
target = (unsigned long) ( (maxZ+1) * (maxY+1) /50 . 0) ; 
target++; 

// Get increments to march through data 

Inputs [0] ->GetContinuousIncrements (outExt, inlncX, inlncY, inlncZ) ; 
outData->GetContinuousIncrements (outExt , outlncX, outlncY, outlncZ) ; 

outPtr = (float*) outData->GetScalarPointerForExtent ( outExt ); 

// Fill in the Hessian pointers 

Hessian [IXY] = (T*) Inputs [ IXY] ->GetScalarPointerForExtent ( outExt ) 
Hessian [IXZ] = (T*) Inputs [ IXZ] ->GetScalarPointerForExtent ( outExt ) 
Hessian [IYZ] = (T*) Inputs [ IYZ] ->GetScalarPointerForExtent ( outExt ) 

Hessian [IXX] = (T*) Inputs [ IXX] ->GetScalarPbinterForExtent ( outExt ) 
Hessian [IYY] = (T*) Inputs [ IYY] ->GetScalarPointerForExtent ( outExt ) 
Hessian [IZZ] = (T*) Inputs [ IZZ] ->GetScalarPointerForExtent ( outExt ) 

Hessian [I] = (T*) Inputs [I] ->GetScalarPointerForExtent ( outExt ); 

T lowThresh, upperThresh; 



// Make sure the replacement values are within the output scalar range 



if (self ->GetLowerIntensityThresh( ) < (float) Inputs [ 0] ->GetScalarTypeMin ( ) ) 
{ 

lowThresh = (T) Inputs [ 0 ] ->GetScalarTypeMin () ; 
} 

else 
{ 

lowThresh = (T) self ->GetLowerIntensi tyThresh () ; 
} 

if (self ->GetUpperIntensi tyThresh ( ) > (float) Inputs [0] ->GetScalarTypeMax ( ) ) 
{ 

upperThresh = (T) Inputs [ 0] ->GetScalarTypeMax () ; 
} 

else 
{ 

upperThresh = (T) self ->GetUpperIntensi tyThresh () ; 
} 



// Loop through ouput pixels 
for (idxZ = 0; idxZ <= maxZ; idxZ++) 
{ 

for (idxY = 0; ! self ->AbortExecute && idxY <= maxY; idxY++) 
{ 

if (!id) 
{ 

if ( ! ( count % target) ) 
{ 

self ->UpdateProgress (count/ (50 . 0*target ) ) ; 
} 

count ++; 
} 

for (idxX = 0; idxX <= maxX; idxX++) 
{ 

for ( idxC = 0; idxC < numC; idxC++ ) 
{ 

if ( (* (Hessian[I] ) ) >= lowThresh && {* (Hessian [ I] ) ) <= upperThresh) 
{ 

// Fill in the Hessian 
qmatrix[0] [0] = (float) 
qmatrix[0] [1] = (float) 
qmatrix[0] [2] = (float) 

qmatrix [ 1 ] [ 0 ] = ( float ) 
qmatrix [ 1 ] [ 1 ] = ( float ) 
qmatrixfl] [2] = (float) 

qmatrix[2] [0] = (float) 
qmatrix[2] [1] = (float) 
qmatrix [2] [2] = (float) 



(* (Hessian [IXX] ) ) 
(* (Hessian [IXY] ) ) 
(* (Hessian [IXZ] ) ) 

(* (Hessian [IYX] ) ) 
(* (Hessian[IYY] ) ) 
(* (Hessian[IYZ] ) ) 

(* (HessianflZX] ) ) 
(* (Hessian[IZY] ) ) 
(* (HessianfIZZ] ) ) 



int status = 

vtkMath: : Jacobi (qmatrix, raw_eigenvalue , raw_eigenvector ) ; 
if ( i status ) { 

vtkGenericWarningMacro ( << " vtklmageHessianResponse : Jacobi 
returned error" ) ; 

1123 = 0.0; 



} 

else 
{ 



SortVectorsByMagnitude (raw__eigenvalue, raw_eigenvector , eigenvalue, eigenvector) ; 
1123 = (*RFP) (self , eigenvalue, eigenvector) ; 
} 

} 

else 
{ 

1123 = 0.0; 
} 

*outPtr = 1123; 

self->GetScalarData ( ) ->InsertNextTuplel (eigenvalue [myeigenvec] ) ; 

self ->GetVectorData ( ) ->InsertNextTuple3 (eigenvector [myeigenvec] [0] , 

eigenvector [myeigenvec] [1] , 
eigenvector [myeigenvec] [2] ) ; 

for ( i = 0; i < 7; i + + ) 
{ 

++Hessian [ i ] ; 
} 

++outPtr; 
} 

for ( i = 0; i < 7; i++ ) 
{ 

Hessian [i] += inlncX; 
} 

outPtr += outlncX; 

} 

outPtr += outlncY; 
for ( i = 0; i < 7; i++ ) 
{ 

Hessian [i] += inlncY; 
} 

} 

outPtr += outlncZ; 
for ( i = 0; i < 7; i++ ) 
{ 

Hessian [i] += inlncZ; 
} 

} 

for (i = 0; i < 3; i++) 
{ 

delete [ ] qmatrix [ i ] ; 

delete [ ] raw_eigenvector [ i ] ; 

} 

delete [] qmatrix; 
delete [ ] raw__eigenvector ; 
delet e [ ] raw_eigenvalue ; 
delete [ ] eigenvector ; 
delete [ ] eigenvalue ; 

// assign the vectors to the image data 

outData->GetPointData ( ) ->SetVectors (self ->GetVectorData ( ) ) ; 
//outData->GetPointData ( ) ->SetScalars ( self ->GetScalarData ( ) ) ; 



self->GetScalarData ( ) ->SetName( "eigenvalues" ) ; 
outData->GetPointData ( ) ->AddArray ( self ->GetScalarData ( ) ) ; 



// 

// This method is passed a input and output data, and executes the filter 
// algorithm to fill the output from the input. 

//It just executes a switch statement to call the correct function for 
// the datas data types. 

void vtklmageHessianResponse: : ThreadedExecute (vtklmageData **inData, 

vtklmageData *outData , 
int outExt[6], int id) 

{ 

int i , j ; 
bool valid; 
int * outExtent; 
int dataType; 

if ( GetNumberOflnputsO != 7 ) 
{ 

vtkErrorMacro ( « "Execute: vtklmageHessianResponse requires 7 inputs" ); 

return; 

} 

vtkDataObject **Inputs = GetlnputsO; 

outExtent = Inputs [ 0 ] ->GetWholeExtent ( ) ; 

dataType = ( (vtklmageData*) Inputs [0] ) ->GetScalarType ( ) ; 

valid = 1; 

for ( i = 0; i < 7; ++i ) 
{ 

vtklmageData* in; 

in = (vtklmageData*) Inputs [i]; 

//Do all of our checks 

valid = valid && ( dataType == in->GetScalarType ( ) ); 
if ( !valid ) 
{ 

vtkErrorMacro ( << "Execute: vtklmageHessianResponse datatype mis-match on 
input" ) ; 

return; 
} 

int *ext = in->GetWholeExtent ( ) ; 
for ( j = 0; j < 7; ++j ) 
{ 

valid = valid && ( ext[j] == outExtent [j] ); 
} 

valid = valid && in->GetNumberOf ScalarComponents ( ) == 
( (vtklmageData*) Inputs [0] ) ->GetNumberOf ScalarComponents ( ) ; 
if ( ivalid ) 
{ 

vtkErrorMacro ( << "Execute: vtklmageHessianResponse extent mis-match on 
input" ) ; 

return; 
} 

} 



switch ( inData [ 0 ] ->GetScalarType ( ) ) 
{ 

vtkTemplateMacroS (vtklmageHessianResponseExecute, this , 

outData, outExt, id, static_cast<VTK_TT*> (0) ) ; 

default : 

vtkErrorMacro(<< "Execute: Unknown ScalarType" ) ; 
return; 

} 



void vtklmageHessianResponse: :PrintSelf (ostream& os, vtklndent indent) 
{ 

vtklmageMultiplelnputFilter : : PrintSelf (os , indent ) ; 

os « indent « "Gammal2: " « this->Gammal2 « "\n"; 
os << indent << "Gamma23: " « this->Gamma23 << "\n"; 
os << indent « "Alpha: " « this->Alpha << ,f \n"; 
if <this->ResponseType == VTK_HES SI ANDRES PONSE_LINE) 
{ 

os << indent << "ResponseType: LINE" << "\n" ; 
} 

else if (this->ResponseType == VTK_HESSIAN_RESPONSE_LINE ) 
{ 

os << indent << "ResponseType: SPHERE" << "\n"; 
} 

else 
{ 

os << indent << "ResponseType: UNKNOWN" « "\n"; 
} 

} 



This Page is Inserted by IFW Indexing and Scanning 
Operations and is not part of the Official Record 

BEST AVAILABLE IMAGES 

Defective images within this document are accurate representations of the original 
documents submitted by the applicant. 

Defects in the images include but are not limited to the items checked: 

□ BLACK BORDERS 

□ IMAGE CUT OFF AT TOP, BOTTOM OR SIDES 

□ FADED TEXT OR DRAWING 

□ BLURRED OR ILLEGIBLE TEXT OR DRAWING 

□ SKEWED/SLANTED IMAGES 

□ COLOR OR BLACK AND WHITE PHOTOGRAPHS 
□yGRAY SCALE DOCUMENTS 

3 LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENCED) OR EXHIBIT(S) SUBMITTED ARE POOR QUALITY 

□ OTHER: 

IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



